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Part Il of the Houston Longitudinal Study on the Transition to College and Work (HLS) examined potential 
indicators of college enrollment school and district staff might use to identify and support students at risk of 
not attending college. The study used administrative data from the Houston Independent School District (HISD) and 
tracked two cohorts of seventh-grade students in fall 2007 and fall 2008 for six years and into the first semester of 
college. Three potential indicators of college enrollment were examined: 


e Chicago: Designed to predict high school graduation’; based on earning six course credits — the minimum to advance 
to the next grade in HISD — and having at most one semester F in a core subject (English, math, science, or social 
studies) 


e Houston Education Research Consortium (HERC): Designed to predict college enrollment (see Appendix A for 
information on its origins); based on having an attendance rate of 90% or higher, having a B-average (80% or higher), 
and earning at least 0.5 advanced course credits 


e State: Designed to predict college enrollment by the Texas Education Agency; based on meeting the benchmark on 
the English/language arts and mathematics tests 


The study found the HERC indicator was more effective at predicting college enrollment in later grade levels than the 
Chicago and state indicators. In addition, a substantial share of students did not meet the Chicago and HERC indicators 
during the ninth-grade year. The state indicator showed different patterns, perhaps because test scores fluctuated little 
during middle and high school and might not reflect changes in student motivation or effort. 


Key Findings 


e The Chicago indicator (Allensworth & Easton, 2005), designed to predict high school dropout?, wasless 
effective at predicting college enrollment than the HERC and state (Texas Education Agency, 2007) 
indicators, which were designed to predict college enrollment. 


e The HERC indicator was more effective than the Chicago indicator at predicting non-college enrollment, 
particularly in ninth and 11th grade. 


e The three potential indicators underestimated the college enrollment rates of white, Asian, and non- 
economically disadvantaged students, but were more accurate for black, Hispanic, and economically 
disadvantaged students. 


e Students were most at risk of not meeting the Chicago and HERC indicators during the ninth grade. 


e Incontrast, the state indicator did not show the same ninth-grade pattern likely because it was based on 
test scores, which might vary little over time or neglect to capture changes in student motivation and 
effort. 


1The Chicago indicator was tested to see whether a high school graduation indicator could also serve as a potential college enrollment 
indicator. 
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Background 


In the state of Texas, only 29 percent of eighth-grade students went on to complete a postsecondary 
credential, ranging from a certificate to a doctorate, within 12 years of high school graduation (Kinder 
Institute for Urban Research, 2018). In response to the low college completion rate, policymakers made 
higher education a priority by aiming to raise the share of young adults with a postsecondary education 
(Texas Higher Education Coordinating Board, 2015) and integrating measures of college readiness into the 
state accountability system (Texas Education Agency, 2018). 


As policymakers set goals for postsecondary attainment, school and district practitioners might seek new 
strategies to ensure students are prepared to enroll in college. Although much is known about early 
warning indicators for both high school graduation and dropout (see Bowers, Sprott, & Taff, 2013 fora 
review), college enrollment is a different concept. If schools and districts could identify an indicator that 
could both predict whether a student would enroll in college as well as when they might be at risk of not 
achieving that milestone, then practitioners might be able to use that information to provide additional 
support to students and get them back on-track to college. 


In this study, three potential indicators of college enrollment that might be considered in an early warning 
system were examined (see Table 1 for details): 


1. Chicago: Designed to predict high school graduation’; based on earning six course credits — the 
minimum to advance to the next grade in HISD — and having at most one semester F in a core 
subject (English, math, science, or social studies) 


2. HERC: Designed to predict college enrollment (see Appendix A for information on its origins); 
based on having an attendance rate of 90% or higher, having a B-average (80% or higher), and 
earning at least 0.5 advanced course credits 


3. State: Designed to predict college enrollment by the Texas Education Agency; based on meeting 
the benchmark on the English/language arts and mathematics tests 


Research Questions 


1. How well did three potential early warning indicators of college enrollment predict college 
enrollment? Were there differences by gender, race/ethnicity, and socioeconomic status (SES)? 


2. During which grade levels were students at risk of not achieving these potential indicators? Were 
there differences by gender, race/ethnicity, and SES? 


2 A prior HERC research brief entitled Evaluating High School Dropout Indicators and Assessing Their Strength examined 
predictors of high school dropout among students in HISD. The brief is available for download at 
https://kinder.rice.edu/research/evaluating-high-school-dropout-indicators-and-assessing-their-strength. 

3 The Chicago indicator was tested to see whether a high school graduation indicator could also serve as a potential college 
enrollment indicator. 


Key Findings 


Research Question 1: How well did three potential early warning indicators of 
college enrollment predict college enrollment? Were there differences by 
gender, race/ethnicity, and socioeconomic status (SES)? 


The following section assesses the predictive power of potential indicators of college enrollment. The goal 
was to identify measures that were: 


e Associated with enrollment 
e =Available in administrative data 
e =Relatively easy for a school or district practitioner to calculate and assess 


Each measure was examined for how accurately it predicted enrollment for all students and subgroups. 
For example, it might be possible a given measure was more accurate in predicting the outcomes of 
females than males. If so, practitioners ought to be aware of the measure’s limitations if they decide to 
use it to identify students for intervention. 


Data 


The study used data from HISD, made available through the Houston Education Research Consortium 
(HERC), from 2007-2014. The data contained information on students’ demographic, socioeconomic, 
behavioral, and academic characteristics. They were also matched to records from the National Student 
Clearinghouse, which provided information on whether a student attended a college or university. 


To answer the first research question, two cohorts of seventh-grade students during fall 2007 and fall 
2008 were tracked for six school years and into the first semester of college. Since the goal of the analysis 
was to determine how early warning indicators predicted college enrollment, the sample was limited to 
students who had no missing data on enrollment or the potential indicators tested. Native American 
students were excluded due to small sample size.’ The final sample size consisted of 12,001 students. 


* Limiting the sample to students who had data available to generate the three potential indicators in three grades (i.e., not 
missing data from the course grades, attendance, or TAKS files) might bias estimates of college enrollment and the potential 
indicator tests. However, these limitations were considered acceptable since college enrollment data were only available for high 
school graduates. Additionally, nearly all missing data were due to students in the PEIMS files not matching to the course grades, 
attendance, and TAKS files. This was likely due to student mobility. A missing data analysis is available from the authors upon 
request. 


Key Findings 


Three potential indicators were calculated and analyzed at three grade levels: seventh, ninth, and 11th. 
These grade levels were the focus of the study because: 


e Seventh grade was often the grade level before schools sorted students by measured and 
perceived ability into different core content courses (e.g., eighth-grade algebra versus pre- 
algebra). 


e Ninth grade was the first year of high school. 


e Eleventh grade was the grade level before students began applying to college and typically when 
they first took college entrance exams like the SAT. During the years analyzed, it was also the final 
grade of state accountability testing. 


Measures 


The outcome measured whether a student enrolled in a college or university the fall following high school 
graduation> — fall 2013 for the 2007-2008 seventh-grade cohort and fall 2014 for the 2008-2009 
seventh-grade cohort — using three potential indicators of college enrollment (Chicago, HERC, state) at 
three grade levels (seventh, ninth, 11th). Although college readiness is multidimensional (Conley, 2010), 
the researchers developed potential indicators of enrollment that were based on measures readily 
available in administrative datasets and might be relatively easy for a practitioner to understand and 
generate on their own (See Table 1). The Chicago indicator was designed to predict high school 
graduation, but it was tested here to determine whether a high school graduation indicator could also 
serve as a potential college enrollment indicator. The indicator was based on whether a student earned 
enough course credits to advance to the next grade level and had no more than one semester grade of F 
in a core subject. The HERC indicator was designed to predict college enrollment (see Appendix A for 
information on its origins) and was based on attendance, average course grades, and advanced course 
credits earned. The state indicator was designed to predict college enrollment and was based on the 
College-Ready Graduates measure in the Texas Education Agency’s accountability system, which was 
based on test scores. 


Methods 


Logistic regression models predicted college enrollment using the Chicago, HERC, and state indicators at 
grades seven, nine, and 11. Subsequent models accounted for basic student background characteristics 
(age, gender, race/ethnicity, English learner, special education, economic disadvantage; details on these 
variables are available in Appendix B), as well as cohort and school fixed-effects (dummy variables for 
each cohort, dummy variables for each school attended in seventh, ninth, or 11th grade). After estimating 
the models, predicted probabilities were calculated. These probabilities were used to determine whether 
the model accurately predicted a student’s college enrollment. 


5The analyses did not distinguish between less-than-two-year, two-year, and four-year college enrollment. Since the academic 
requirements of these types of colleges might differ, distinct indicators might need to be developed to correctly predict these 
specific types of enrollment. Please contact the authors for additional information. 


Key Findings 


Table 1. Potential College Enrollment Indicators Used in the Analyses 


Potential 


farelercweyg 
Chicago 


Developer 


University of 
Chicago 
Consortium on 
School Research 


The indicator was developed to predict high school 
graduation (Allensworth & Easton, 2005).° At the end 
of ninth grade, students were classified as “on-track” 
if they earned five course credits — the minimum 
number to advance to 10th grade in Chicago Public 
Schools — and had at most one semester grade of Fin 
a core subject (English, math, science, or social 
studies). Despite its simplicity, the indicator was 
considered an accurate predictor of high school 
graduation (Bowers, Sprott, & Taff, 2013). 


Definition Used 
in Study 


Earned six course 
credits — the 
minimum to advance a 
grade in HISD — and 
had at most one 
semester grade of F in 
a core subject (English, 
math, science, or 
social studies)’ 


HERC 


Rice University 
Houston Education 
Research 
Consortium 


By reviewing relevant literature and analyzing HERC 
data, the goal was to develop an indicator that was 
relatively easy for a school or district practitioner to 
calculate and assess (like Chicago’s) and was more 
tied to college enrollment; additional details are 
available in Appendix A. Three components were 
considered — attendance rates, grades earned in 
courses taken, and credits earned in advanced 
courses. Advanced courses referred to pre-Advanced 
Placement (AP), pre-International Baccalaureate (IB), 
AP, IB, or academic dual credit® courses. 


Had an attendance 
rate of 90% or higher, 
had a B-average (80% 
or higher), and earned 
any advanced course 
credits 


State 


Texas Education 
Agency Academic 
Excellence 
Indicator System 


During the 2006-2007 school year, the state of Texas 
incorporated a measure called College-Ready 
Graduates into its annual school performance reports. 
The measure identified students who met 
benchmarks on the state English/language arts test, 
state mathematics test, SAT test, or ACT test. 


Met the benchmark on 
the English/language 
arts and mathematics 
tests?10 


6 The Chicago indicator was tested to see whether a high school graduation indicator could also serve as a potential college 
enrollment indicator. 
7 Each half-credit was defined as passing a semester-long course (grade of 69.5% or above). Therefore, six course credits 

corresponded to passing 12 semester-long courses. A semester-long course with a grade below 69.5 percent counted as a 
semester grade of F. 


8 Academic dual credit courses were dual credit courses that were not Career & Technical Education. 


° Although the College-Ready Graduates measure incorporated multiple tests, the study focused on the state English/language 
arts and mathematics tests since the goal was to generate annual measures. Students typically did not take the SAT or ACT 


annually. 


10 English/language arts and mathematics cutoffs were based on the 11th-grade tests — 2200 for English/language arts and 2200 
for mathematics. The numerical cutoffs were identical for the two subjects and did not change over time. Cutoffs identified in the 
test score distribution were applied to tests in other grades. For example, the mathematics cutoff for the 11th-grade test in 
2007-2008 corresponded to a standardized score of -0.24. In the same year, a standardized score of -0.24 corresponded to a raw 
score of 2175.68 on the seventh-grade mathematics test. This raw score was the cutoff used to determine whether a student 
met the mathematics standard in seventh grade. Please contact the authors for additional information. 


Key Findings 


The Chicago indicator was less effective at predicting 
college enrollment than the HERC and state indicators. 


Results 


Table 2 presents the results from tests of the three potential indicators of college enrollment in grades 
seven, nine, and 11. In each panel, the first three rows show how well each potential indicator predicted 
college enrollment without accounting for background characteristics, cohort, or school effects. In 
seventh grade (Panel A), all three potential indicators were positively correlated with enrollment, 
although the HERC indicator showed the highest correlation (r = 0.45). The HERC and state indicators 
explained more of the variance in enrollment (Pseudo-R? = 0.06) than the Chicago indicator. The last 
three columns show the share of correct predictions. Overall, without background characteristics, cohort, 
or school effects, the three potential indicators correctly predicted nearly two-thirds of students’ 
enrollment outcomes. However, the HERC and state indicators performed slightly better than the Chicago 
indicator. These findings made sense given the HERC and state indicators were developed with college in 
mind, whereas the Chicago indicator was designed for high school graduation. 


The differences between the potential indicators in seventh grade became more pronounced when 
looking at the correct positive and negative predictions. The Chicago indicator was highly accurate in 
terms of predicting enrollment (92%), but much less accurate in predicting non-enrollment (21%); that is, 
it assumed 79 percent of non-enrollees went to college. Of course, some non-enrollees might have 
intended not to go to college. Nevertheless, as the state integrates measures of college readiness into its 
accountability system (Texas Education Agency, 2018), practitioners may want to target potential non- 
enrollees for intervention. If the potential indicators tell a practitioner a student will likely attend college 
when they actually will not, then there is a missed opportunity. The Chicago indicator might predict non- 
enrollment poorly because it was too lenient, the bar was set too low. While the HERC and state 
indicators’ correct positive prediction rates (79% and 69%, respectively) were lower than the Chicago 
indicator’s, their correct negative prediction rates were higher (49% and 60%, respectively).** What this 
meant was these potential indicators were able to identify correctly half or more non-enrollees, which 
might be information useful for intervention. 


The next two sections of the seventh-grade panel add in background characteristics, then cohort and 
school effects. The addition of these measures increased the explained variation and the correct overall 
prediction rate. The most notable change occurred with the potential indicators’ share of correct positive 
and negative predictions; they began to resemble one another. 


11 As a robustness check, potential indicators from eighth grade were tested and showed patterns similar to those from seventh 
grade. Specifically, the state indicator performed slightly better than the HERC indicator in terms of correct negative predictions. 
It was possible attendance, grades, and advanced course credits in middle school were less predictive than they were in high 
school. Therefore, practitioners might consider relying more on test score indicators in early grades and indicators like the HERC 
indicator in later grades. 


Key Findings 


The HERC indicator was more effective than the Chicago 
indicator at predicting non-college enrollment, 


particularly in ninth and 11th grade. 


In sum, while there were similarities between the three potential seventh-grade indicators in correct 
overall predictions, the Chicago indicator demonstrated slightly worse performance. Moreover, it was 
particularly inefficient at predicting non-enrollment on its own, and to achieve correct prediction rates 
similar to those of the HERC and state indicators, control variables for sociodemographic characteristics 
and/or cohort and school effects must be added. This suggests the Chicago indicator may not be effective 
as a potential indicator of college enrollment. In contrast, the state and, to a lesser extent, the HERC 
indicators had correct negative prediction rates that were substantially higher and less affected by 
additional variables. If a practitioner wants to identify a potential indicator that can help identify potential 
non-enrollees, the HERC and state indicators alone may perform fairly well, whereas the Chicago 
indicator may require a more sophisticated analysis to perform as well. 


Panels B and C of Table 2 show the same statistics using the Chicago, HERC, and state indicators from the 
ninth and 11th grades. Correlations and explained variances between the potential indicators and college 
enrollment were slightly higher in these later grades. This may be tied to the fact that the potential 
indicators in grades nine and 11 were closer to the time of college enrollment. Patterns of correct overall 
predictions in grades nine and 11 were similar to those reported in seventh grade. However, one key 
change was that without control variables for sociodemographic characteristics, cohort, and school 
effects, the rate of correct negative predictions dropped for the state indicator and increased for the 
HERC indicator. In addition, the correct positive prediction rate increased for the state indicator and 
declined for the HERC indicator. Low test scores during high school might be less consequential to college 
enrollment than chronic absenteeism, low grades, and lack of advanced credits. These latter measures 
might capture unobservable factors like low motivation and effort, which might drive students away from 
college and might not be easily reflected on a standardized test. It appeared the HERC indicator’s ability 
to predict non-enrollment — a feature that might be useful to a practitioner — strengthened over time, 
while the state indicator’s ability to do so weakened. 


Key Findings 


The three potential indicators underestimated the college 
enrollment rates of white, Asian, and non-economically 
disadvantaged students, but were more accurate for black, 


Hispanic, and economically disadvantaged students. 


Practitioners may consider whether the potential indicators predict college enrollment equally well for 
different groups of students. In short, the answer was no. In additional analyses, the correct overall 
prediction rates for female, white, Asian, and non-economically disadvantaged students slightly exceeded 
the rates for male, black, Hispanic, and economically disadvantaged students. However, despite the 
higher correct overall prediction rates, female, white, Asian, and non-economically disadvantaged 
students showed low correct negative prediction rates. For these groups of students, the models did not 
capture non-enrollment accurately. Students from more advantaged groups who did not meet a given 
potential indicator were more likely to enroll in college than students from less advantaged groups who 
also did not meet a potential indicator. There might be external factors (e.g., parental education, social 
capital) driving the enrollment of students from more advantaged groups the models were unable to 
capture. These factors might push advantaged students into college even if they did not meet the 
potential indicators. Had the models controlled for external factors, predictors of non-enrollment for 
more advantaged groups might have been more accurate. If practitioners wish to use the potential 
indicators for intervention, they ought to be aware of this limitation and gather additional information 
from students before targeting them for resources. Graphs illustrating the correct overall, positive, and 
negative predictions by subgroup are available in Appendix D. Additional details are available from the 
authors upon request. 


For more analyses of the potential indicators, please see Appendices E and F. 


Key Findings 


Table 2. Predictive Power of Potential College Enrollment Indicators, by Grade 


Panel A. Potential Seventh-Grade Indicators 
Potential Indicator Polychoric Pseudo-R? Correct Overall Correct Positive Correct Negative 
Correlation Prediction Prediction Prediction 
Chicago 0.37 0.03 62% 92% 21% 
HERC 0.45 0.06 66% 79% 49% 
State 0.44 0.06 65% 69% 60% 
Control for Sociodemographic Characteristics 
Chicago 0.11 66% 78% 51% 
HERC 0.12 67% 80% 51% 
State 0.12 68% 76% 56% 
Add Cohort and School Fixed-Effects 
Chicago 0.13 69% 80% 54% 
HERC 0.14 69% 78% 56% 
State 0.14 68% 77% 56% 
Panel B. Potential Ninth-Grade Indicators 
Potential Indicator Polychoric Pseudo-R? Correct Overall Correct Positive Correct Negative 
Correlation Prediction Prediction Prediction 
Chicago 0.38 0.03 63% 90% 26% 
HERC 0.51 0.08 66% 61% 73% 
State 0.43 0.05 65% 81% 44% 
Control for Sociodemographic Characteristics 
Chicago 0.11 67% 82% 46% 
HERC 0.14 69% 73% 63% 
State 0.12 67% 81% 50% 
Add Cohort and School Fixed-Effects 
Chicago 0.15 69% 77% 57% 
HERC 0.17 71% 77% 62% 
State 0.15 69% 78% 56% 
Panel C. Potential 11th-Grade Indicators 
Potential Indicator Polychoric Pseudo-R? Correct Overall Correct Positive Correct Negative 
Correlation Prediction Prediction Prediction 
Chicago 0.51 0.07 67% 90% 35% 
HERC 0.56 0.11 68% 64% 74% 
State 0.47 0.07 67% 79% 50% 
Control for Sociodemographic Characteristics 
Chicago 0.14 69% 84% 49% 
HERC 0.16 70% 75% 64% 
State 0.13 68% 80% 53% 
Add Cohort and School Fixed-Effects 
Chicago 0.17 71% 81% 58% 
HERC 0.19 72% 78% 65% 
State 0.16 70% 79% 58% 
Source: HERC multi-year data. 
Note: Sample was limited to non-Native American students who were not missing data. Results came from logistic regression 
models that predicted college enrollment; controlled for the Chicago, HERC, or state indicator in grade seven, nine, or 11; 
controlled for student background characteristics (age, gender, race/ethnicity, English learner, special education, economic 
disadvantage); and controlled for cohort and school fixed-effects (dummy variables for each cohort, dummy variables for each 
school attended in seventh, ninth, or 11th grade). 


Key Findings 


Research Question 2: During which grade levels were students at risk of not 
achieving these potential indicators? Were there differences by gender, 
race/ethnicity, and SES? 


In the first section of this report, three potential indicators were tested for how well they predicted 
college enrollment. In terms of correct overall predictions, findings showed the HERC and state indicators 
performed as well as or slightly better than the Chicago indicator. Differences were more pronounced 
when considering correct positive and negative predictions. While the Chicago indicator was good at 
predicting enrollment for enrollees, it was less adept at predicting non-enrollment for non-enrollees. This 
might be expected since the Chicago indicator was designed to predict high school graduation, not 
college enrollment. Regardless of the potential indicator a school or district decides to use in an early 
warning system, practitioners may wish to know when a student may be most at risk of falling off-track 
from college, as well as the most appropriate time to intervene. 


Data 


To determine the grade during which students fell off-track, HERC data from 2007-2014 were used. Like 
Research Question 1, the analyses focused on two cohorts of seventh-grade students in fall 2007 and fall 
2008 and followed them for six years, through grade 12. The sample was limited to non-Native American 
students who were not missing data on the Chicago, HERC, and state indicators or on control variables 
used in the statistical models.” The final analytic sample consisted of 17,879 students.*° 


Separate data files were created for the Chicago, HERC, and state analyses, respectively. The files were 
restructured into a person-period format so that each observation represented a student-grade (e.g., 
student A in grade 7, student A in grade 8, student Ain grade 9). This longitudinal format was required for 
the event history analysis, the method used to determine when students were at risk of falling off-track 
and when practitioners ought to intervene.” 


12 Nearly all missing data were due to students in the PEIMS files not matching to the course grades, attendance, TAKS, and 
discipline files. This was likely due to student mobility. A missing data analysis is available from the authors upon request. 

13 |n statistical models, there were actually more than 17,819 observations. The analytic strategy required creating a student-by- 
grade dataset in which there was one row for each student in each grade level. The number of student-by-grade observations 
reported in the tables in Appendix | varied because students exited the sample once they did not meet an indicator. Because 
more students did not meet the HERC indicator than the Chicago indicator, there were fewer student-by-grade observations in 
the HERC indicator analysis (46,228) than the Chicago indicator analysis (63,352). Although more students did not meet the HERC 
indicator than the state indicator, the state indicator analysis (41,936) included fewer student-by-grade observations than the 
HERC indicator analysis because fewer grade levels were included in the state indicator analysis; there was no 12th-grade test, so 
a state indicator could not be developed. Additional details on the data structure are available from the authors upon request. 

4 Because of this restructuring, the actual sample size for the regression models included 63,352 student-grades for the Chicago 
indicator analysis, 46,228 student-grades for the HERC indicator analysis and 41,936 student-grades for the state indicator 
analysis. 
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Measures 


The dependent variables were binary (0/1) and measured whether a student did not meet the Chicago, 
HERC, or state indicator in a given grade level. The key independent variable of interest was a categorical 
measure of the grade level during which a student first did not meet the potential indicator. The 
reference category was seventh grade and the measure determined the grade during which students 
were most at risk of falling off-track in terms of the Chicago, HERC, or state indicators. The coefficients on 
the grade dummies showed whether students were more or less likely to not meet a potential indicator in 
a given grade, as compared to seventh grade. 


Aside from the grade level measure, the statistical models controlled for a variety of student background 
characteristics. Most notably, the models included lagged measures from the sixth grade to account for 
baseline achievement: the total number of credits earned, the number of core courses failed, the 
attendance rate, the average grade percentage, the number of advanced credits earned, 
English/language arts test scores, and mathematics test scores. In addition, the models controlled for age, 
gender, race/ethnicity, English learner, special education, and economic disadvantage from the seventh- 
grade data file. The models also included the number of in-school and out-of-school suspensions since it 
might be possible student behaviors in middle and high school correlated with whether a student was on- 
track to college. These two count variables were allowed to vary over time. Finally, the models accounted 
for differences between the two cohorts as well as differences between schools.”° 


Methods 


Using the student-by-grade data files, event history models that predicted when a student first did not 
meet the potential indicator of college enrollment were estimated. Separate models were estimated for 
the Chicago, HERC, and state indicators. All models controlled for grade, student background 
characteristics, cohort effects, and seventh-grade school effects. Additional details on the statistical 
models are available in Appendix G. 


Students were most at risk of not meeting the Chicago 
and HERC indicators during the ninth grade. 


Results 


Figures 1-3 show the main results from the analyses with all control variables and fixed-effects; full 


regression tables are available in Appendix |. For each potential indicator, two plots are produced. The 
first plots the hazard curve, which shows the share of students who fell off-track (i.e., did not meet the 
Chicago, HERC, or state indicator) in each grade. If the hazard curve for a given grade is higher, then that 


15 |n robustness checks, time-varying school characteristics (percent of economically disadvantaged students, number of 
advanced courses offered, student-teacher ratio, average years of teacher experience) were included. Results were similar to 
those reported and are available from the authors upon request. 
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meant more students fell off-track during that grade and that might be when a practitioner ought to 
intervene. The second plot shows the survival curve, which is the share of students still on-track by the 
end of each grade. This is a cumulative measure. If, for example, the survival curve shows a value of 20 
percent in 12th grade, then that meant only 20 percent of the sample met the potential indicator during 
all six grade levels, seventh through 12th. 


In terms of the Chicago indicator (Figure 1), ninth grade was the year during which a slightly higher 
percentage of students did not meet the potential indicator (13%). Eleven percent of students did not 
meet the potential indicator in grades seven, 10, and 11. This finding was in line with prior work 
demonstrating the ninth-grade year was important for high school graduation (Allensworth & Easton, 
2005). Turning to the survival curve, by the end of high school, approximately 46 percent of students did 
not meet the Chicago indicator at least once and 54 percent met it all six years. 


Figure 1. Falling Off-Track Based on the Chicago Indicator 


Estimated Hazard Probability Estimated Survival Probability 
70% 100% Note: Sample was limited 
to 17,879 non-Native 
60% 
80% American students with 
0% non-missing data. Results 
z 3 came from discrete-time 
ine = 6% ; 
= 10% § hazard models with control 
£ g ‘ . 
5 aa = variables, cohort fixed- 
a 5 effects, and seventh-grade 
a 
0% school fixed-effects. 
20% 
10% oe 
0% 0% 
7 8 9 10 11 12 7 8 9 10 11 12 
Grade Grade 


Source: HERC multi-year data. 


Figure 2 shows hazard and survival plots for the HERC indicator. Results clearly showed ninth grade was 
when a substantial share of students fell off-track; approximately 47 percent did not meet the potential 
indicator during this school year. The survival curve showed a steep drop between grades eight and nine; 
what this meant was a substantial share of students did meet the indicator during the ninth grade. At the 
end of eighth grade, 71 percent of students were still on-track (i.e., met the HERC indicator in both grades 
seven and eight), but by the end of ninth grade, only 37 percent of students were still on-track (i.e., met 
the HERC indicator in grades seven through nine). By the end of high school, only 15 percent of students 
in the sample met the potential indicator all six years; the wide majority did not meet it at least once. 


Finally, Figure 3 shows results for the state indicator. Please note the event history analysis for the state 
indicator excluded grade 12 since state tests were not administered that year. According to the hazard 
curve, seventh grade was the school year during which a higher share of students did not meet the 
potential indicator (24%). The hazard curve flattened out after that point. By the end of 11th grade (the 
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last testing year), 45 percent of students had not met the state indicator one or more times, while 55 


percent met it every single year. 


Figure 2. Falling Off-Track Based on HERC Indicator 
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Note: Sample was limited 
to 17,879 non-Native 
American students with 
non-missing data. Results 
came from discrete-time 
hazard models with 
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Figure 3. Falling Off-Track Based on the State Indicator 
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Key Findings 


The state indicator did not show the same ninth-grade 
pattern. 


The results indicated that according to the Chicago and HERC indicators of college enrollment, the school 
year during which more students fell off-track was ninth grade. Students were less likely to hit key 
benchmarks that year (e.g., earning high grades, completing advanced credits). If school and district 
practitioners wish to increase college enrollment and implement an intervention or provide targeted 
assistance to a specific grade level, then they may consider doing so for ninth-grade students. This 
pattern was not apparent with the state indicator. Seventh grade — the first grade level in the analysis — 
was when the highest share of students missed the state indicator. After that, the hazard curve was flat. 


There might be two interpretations for these findings. First, test score indicators (and raw test scores) 
were quite correlated over time. Although there was an expectation potential indicators from 
neighboring grades would be correlated with one another (since academic performance might not 
change dramatically in a year, for example), the correlation between potential indicators from non- 
sequential grades might be more muted, as well; supplemental analyses confirmed this.*° If test scores 
were less variable over time than the measures underlying the Chicago and HERC indicators (e.g., grades 
earned, credits completed), then that might explain why there was little change in the hazard curve. 


Another explanation for why the state indicator might not follow the same pattern as the Chicago and 
HERC indicators was the Chicago and HERC indicators measured more than student ability. To be sure, to 
earn an Aor B, a student must be skilled; however, they also must be motivated and put in a great deal of 
effort by paying attention in class, completing assignments, and studying for exams. Students might 
experience dramatic changes in their motivation and effort over time if, for example, they disliked their 
teachers, moved to a new school, or had trouble in their home lives. In contrast, standardized tests might 
be less subject to changes in student motivation and effort over time since they were snapshots ofa 
student’s academic ability (i.e., administered during a single week in the school year). It should be noted 
the increased share of students falling off-track in the ninth grade for the Chicago and HERC indicators 
likely might be tied to the transition from middle to high school. This transition would involve a structural 
move (i.e., moving to a new building), encountering new students and teachers, and facing higher 
expectations. These changes might have been challenging for students and disrupted their patterns of 
attendance, grades, and course-taking. 


Aside from the ninth-grade findings, another key observation was the survival curve was steeper for the 
HERC indicator than it was for the Chicago indicator; what this meant was that more students did not 
meet the HERC indicator than the Chicago indicator over time. The Chicago indicator might be considered 
a lenient benchmark of college enrollment; as mentioned, it was designed to predict high school 
graduation. Moreover, while the ninth-grade transition might pose challenges to students in terms of 
passing classes, it might have been disruptive beyond the lowest-achieving students who were most likely 


16 These results are available from the authors upon request. 
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Key Findings 


to drop out. Middle- and high-achieving students might also find the ninth grade challenging. Although 
they might not be at risk of failing a class or being retained, middle- and high-achieving students might 
struggle with their schoolwork, earn lower grades than they were used to, and choose to take easier 
courses. In effect, the HERC indicator represented a higher standard that not only affected the lowest- 
achieving students at risk of high school dropout, but a// students who might find the changes and rigors 
of high school challenging. 


In additional analyses, interaction terms were incorporated to test for differential effects by gender, 
race/ethnicity, and economic disadvantage. These models helped determined whether the grade during 
which students were at risk of falling off-track and not meeting the Chicago, HERC, and state indicators 
varied by student background (See Appendix J). Overall, patterns varied little among subgroups. 
Regardless of demographic background or socioeconomic status, ninth grade was a challenging transition 
in terms of the HERC indicator and, to a lesser extent, the Chicago indicator. Moreover, the state 
indicator showed seventh grade — the first year in the analyses — was when the plurality of students did 
not meet that measure. 


While male, black, Hispanic, and economically disadvantaged students were less likely to meet the 
Chicago, HERC, or state measures, a more troubling finding was that the hazard curves, which traced 
when students were at greatest risk of not meeting a measure, were steeper. This meant male, black, 
Hispanic, and economically disadvantaged students were less likely to meet the Chicago, HERC, and state 
indicators than female, white, Asian, and non-economically disadvantaged students. For example, using 
the HERC indicator, the share of students not meeting the potential indicator in ninth grade was relatively 
low for white and Asian students, 15 and 12 percent, respectively. In contrast, the share of black and 
Hispanic students not meeting the HERC indicator in ninth grade was dramatically higher, 64 and 53 
percent, respectively. There were also large differences by SES: while 23 percent of non-economically 
disadvantaged students did not meet the ninth-grade HERC indicator, 58 percent of economically 
disadvantaged students did not meet it. The ninth-grade transition was challenging for all students, but if 
the HERC indicator might be considered a reliable predictor of college enrollment, then that transition 
could have particularly harmful consequences for black, Hispanic, and economically disadvantaged 
students. 
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Conclusion 


The goal of this study was twofold: 1) to test different potential indicators of college enrollment and 
determine which ones might be useful to practitioners in identifying students for early intervention and 2) 
to examine the potential indicators in an effort to understand when students might be at risk of not 
meeting them and, implicitly, falling off the pathway to college. 


Overall, the assessment of the potential indicators suggested the HERC indicator might be the best 
predictor of college enrollment. Not only did it show high rates of correct overall predictions, but it also 
correctly predicted outcomes for a majority of non-college enrollees, particularly in later grades. (In 
seventh grade, the state indicator correctly predicted non-college enrollment at a higher rate than both 
the HERC and Chicago indicators.) Correctly predicting non-enrollment for non-college enrollees is 
especially important for educational decision-makers. Practitioners may target non-enrollees for early 
intervention and help them plan and prepare for postsecondary education. Correctly predicting outcomes 
for enrollees is important, too — it can help education decision-makers understand where not to 
intervene. 


School and district practitioners must exercise caution when using the potential indicators to predict 
outcomes for students of different backgrounds. The college enrollment rates of white, Asian, and non- 
economically disadvantaged students who did not meet the Chicago, HERC, and state indicators were 
underestimated. These students might have additional supports outside of school, which could not be 
captured in the analysis and might push them into college despite subpar academic performance. 


In addition, the event history analyses suggested, in terms of the Chicago and HERC indicators, ninth 
grade was a challenging time for students. This was the year during which higher shares of students fell 
off-track. However, the findings did not show this pattern with the state indicator, which was based on 
test score cutoffs. Standardized tests, which are highly correlated year-to-year, may not capture 
motivation and effort in school and thereby be less subject to variation caused by external shocks like the 
structural move from middle to high school or changes in friend groups, teachers, and course rigor. 


HISD has implemented a number of innovative programs aiming to improve college readiness among its 

students. For example, during the 2018-2019 school year, the district launched Project Explore, a college 
readiness initiative targeting high-achieving middle-school students. The students in our analyses did not 
have opportunities to participate in this program or HISD’s recent college readiness efforts because they 
did not yet exist, so it is possible outcomes for future cohorts will be different. 


It is beyond the scope of this report to suggest specific strategies to district practitioners, but, based on 
the findings, targeting the ninth-grade transition is key to ensuring students stay on-track to college. 
Additionally, the potential indicators tested may be part of those efforts if they can help teachers, 
counselors, and administrators identify students in need of additional resources and supports. 
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